This pr intends to unify all diverging TokenBucket implementations. waku-org/nwaku#3543
With different needs arising from nim-waku for DOS protection of req/resp protocol a slightly different implementation is added to nim-waku, copied and adopted from nim-chronos.
Now nim-chat-sdk has new requirements.
For to be future proof we decided to go back to the roots and fullfill all these needs in single place.
Also libp2p use was taken into account, so no interface is broken and expected token utilization is similar to original implementation.

What's done:

• Original implementation of TokenBucket in nim-chronos has an issue (hence Short replenish test was failing in Ci and got skipped), when there were enough time passed between TokenBucket initialization and first update.
  Because the initial period is not calculated from the first consume but from init, could cause overuse of tokens.
  This bug is fixed here.
• Waku implemented a different update methods.
  • There were a strict mode added, which allows only refill after full period passed, not in between. > This mode also needed in nim-chat-sdk
  • Other compensating mode was following a burst ballancing calculation, but different from original implementation, allowing overuse of tokens after silent period of time to some extent.
  • In this PR I revised this approach on both sides and refined the original implementation a bit to be smoother with small bursts within fill period and better utilize tokens compared to the possible available tokens during the given time frame. This will satisfy nim-waku compensating mode requirements also - thus no need to distinguish that with a third mode.
  • Now this Ballanced mode is the default for TokenBucket while user can decide to go for Strict replenish mode at creation.
• nim-chat-sdk and nim-waku additional interface needs are added without compromising backward compatibility.
• tests for TokenBucket are extended to cover all the changes while old ones kept to ensure backward compatibility.

In follow up PR may consider to move more rate limiting features from nim-waku to chronos...

Documentation and comparison to previous implementation

TokenBucket — Usage Modes (Overview)

TokenBucket provides several usage modes and patterns depending on how you want to rate-limit:

• Continuous mode (default):

  • Mints tokens proportionally to elapsed time at a constant rate (capacity / fillDuration), adding only whole tokens.
  • When the bucket is full for an interval, the elapsed time is burned (no “credit banking”).
  • If an update would overfill, budget is clamped to capacity and leftover elapsed time is discarded; lastUpdate is set to the current time.
  • Nanosecond-level accounting for precise behavior.

• Discrete mode:

  • Replenishes only after a full fillDuration has elapsed (step-like refill behavior).
  • Before the period boundary, budget does not increase; after the boundary, budget jumps to capacity.
  • Use when you need hard period boundaries rather than proportional accrual.

• Manual-only replenish (fillDuration = 0):

  • Disables automatic minting; tokens can only be added via replenish(tokens).
  • Replenish is capped at capacity and wakes pending consumers.

• Synchronous consumption: tryConsume(tokens, now)

  • Attempts to consume immediately; returns true on success, false otherwise.
  • If consuming from full, lastUpdate is set to now (prevents idle-at-full credit banking in Continuous mode).

• Asynchronous consumption: consume(tokens, now) -> Future[void]

  • Returns a future that completes when tokens become available (or can be cancelled).
  • Internally, the waiter is woken around the time enough tokens are expected to accrue, or earlier if replenish() is called.

• Capacity and timing introspection: getAvailableCapacity(now)

  • Computes the budget as of now without mutating bucket state.

• Manual replenishment: replenish(tokens, now)

  • Adds tokens (capped to capacity), updates timing, and wakes waiters.

The sections below illustrate Continuous semantics with concrete timelines and compare them with the older algorithm for context.

TokenBucket Continuous Mode — Scenario 1 Timeline

Assumptions:

• Capacity C = 10
• fillDuration = 1s (per-token time: 100ms)
• Start: t = 0ms, budget = 10, lastUpdate = 0ms

Legend:

• Minted tokens: tokens added by Continuous update at that step (TA)
• Budget after mint: budget after minting, before the consume at that row
• Budget after consume: budget left after processing the request at that row
• LU set?: whether lastUpdate changes at that step (reason)

Only request events are listed below (no passive availability checks):

Notes:

• When an update would overfill the bucket, it is clamped to capacity and lastUpdate is set to the current time; leftover elapsed time is discarded.
• Consuming from a full bucket sets lastUpdate to the consume time (prevents idle-at-full credit banking).

Consumption Summary (0–3s window)

Per fillDuration period (1s each):

Total consumed over 3 seconds: 15 + 11 + 10 = 36 tokens.

Old Algorithm (Pre-unification) — Scenario 1 Timeline

Reference: old TokenBucket from master (chronos/ratelimit.nim) before the unification.

Key behavioral differences vs current Continuous mode:

• No LU reset when consuming from a full bucket (LU stays unchanged on consume).
• No explicit burn of leftover elapsed time when capacity is reached; fractional leftover time is retained via LU based on minted tokens.

Assumptions and inputs are identical to the table above:

• Capacity C = 10, fillDuration = 1s (100ms per token)
• Request-only events at: 0ms (7), 200ms (5), 650ms (3), 1200ms (6), 1800ms (5), 2100ms (10), 2600ms (10)
• Start: t = 0ms, budget = 10, lastUpdate = 0ms

Consumption Summary (0–3s window, old algorithm)

Total consumed over 3 seconds (old): 36 tokens.

Diff Notes — Continuous vs Old

• Consume from full:

  • Continuous: sets lastUpdate to the consume time before subtracting.
  • Old: does not modify lastUpdate on consume.

• When the bucket is full during an elapsed interval:

  • Continuous: burns all the elapsed time by setting lastUpdate = currentTime (no hidden time credit).
  • Old: advances lastUpdate only by the time used to mint whole tokens, leaving fractional leftover time to carry to the next update.

• Hitting capacity mid-update (overfill path):

  • Continuous: clamps to capacity and sets lastUpdate = currentTime, discarding leftover elapsed time.
  • Old: clamps to capacity but sets lastUpdate = lastUpdate + usedTime (time to mint the whole tokens), retaining the leftover fractional part of the elapsed interval.

• Time resolution:

  • Continuous: nanosecond-based accounting.
  • Old: millisecond-based accounting. Sub-ms differences and rounding may diverge.

Practical impact:

• Continuous avoids multi-call burst inflation at a single timestamp and prevents banking implicit credit during idle-at-full periods.
• In this scenario with millisecond-aligned times, total consumption over 3 seconds is the same (36 tokens), but LU handling differs and becomes visible with sub-token leftover time, long idle while full, or overfill updates.

Comparison Examples

High-rate single-token requests (Continuous)

Settings:

• Capacity C = 10, fillDuration = 10ms (per-token time: 1ms)
• Window to observe: 0–40ms (4 full periods)
• Requests are 1 token each; batches occur at specific timestamps.

We show how the bucket rejects attempts that exceed the available budget at each instant, ensuring no more than capacity + minted tokens are usable in any time frame. Over 0–40ms, at most 10 (initial capacity) + 4 × 10 (mint) = 50 tokens can be consumed.

Request batches and outcomes:

Totals over 0–40ms:

• Attempted: 12 + 7 + 15 + 3 + 25 + 9 + 3 + 20 = 94 requests
• Accepted: 10 + 5 + 5 + 2 + 8 + 9 + 2 + 9 = 50 tokens (matches 10 + 4×10)
• Rejected: 94 − 50 = 44 requests

Why the rejections happen (preventing overuse):

• At any given instant, you can only consume up to the tokens currently in the bucket.
• Between instants, tokens mint continuously at capacity / fillDuration = 1 token/ms; the table shows how many become available just before each batch.
• When a batch demands more than available, the excess is rejected (or would be queued with consume()), enforcing the rate limit.
• Over any observation window, the maximum consumable tokens = initial available (up to capacity) + tokens minted during that window; here, that cap is 10 + (40ms × 1/ms) = 50.